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ABSTRACT , 

This report is a summary of the review of the Defense 
Language Institute (DLI) that was made by the General Accounting 
Off fee (GAO) at the request of a member of Congress. The objectives 
of the study were to identify and analyze significant inj;ernal DLI 
problems that diminish the overall effectiveness of the language 
training provided there. Accordingly, three issues were examined and 
reported:- (1) course development activities, (2) managemcrnt* of 
classroom instruction, and <3) evaluation of graduates and training. 
After at) introduction that describes the concern of the DLI and the 
objective, scope and methodology of the study, the report is djfyided 
into three sections corresponding to the three issues examined in the 
study. The investigation recommended that the Institute: (1) replace 
outdated materials in basic courses,* (2) upgrade the management of * 
classroom instruction, and (3) better assess the effectiveness of its 
training mission. Appendices include information on DLI student 
enrollment and instructors by language as of September 1981,' a review 
oi previous studies of operations and training at DLI^ and the 
Foreign Service Institute's language skill level descriptions. ^ 
(Author/AMH) . 
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The defense Language institute, Foreign 
Language Center, manages the largest for- 
eign-language-training effort in the United 
States. The vast majority of trained lan- 
guage personnel supports the defense intel- 
ligence mission, and such training is cbnsid- 
ered vital to preserving national security. 

Because of a series of problems at the Insti- 
tute, dko concludes that changes are neces- 
sary to improve thb quality of instruction. 
GAO recommends that the Institute (1) 
replace outdated basic course materials, (2) 
upgrade the management 4>f classroom in- 
struction, and (3) bettpr assess the effec- 
tiveness of its training mission. ^ 

DOD responded that it was taking corfec%^^ 
tive action and Kad, in some cases, made 
improvements in its language training. 
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UNITED STATES GENERAL ACCOUNTING OFFICE 
' WASHINGTON, D.C. 2084a 



COMKNSATIOM OIVIttON 



.B-205861 



The Honorable 'John 0. -Mirsh, Jr. 
The Secretary^ of the Army 

Dear Mr. Secretary! 



Attention: 



The Inspector General 
DAIG-AI * . ' i 



Pursuant to a request from Congressman Le<^E. Panetta, we 
examined the management operations of the Defense Language. Insti- 
tute, Foreign Language Center, at Monterey, California, The re- 
port discusses what ye believe are the significarit internal prob-^ 
lems which diminish the overall effectiveness of languac^e training 
at the Institute, / ^ ^ ' 

The report contains recommendations wh^ich require specific 
action on your part,- As you know., section 236 of tjie Legislative , 
Reorganization Act of. 1970 requires the head pf a Federal-agency 
to submit a written statement on actions taken on Qur recommenda- 
tions to the House Coirtmittee on Government Operations and the 
Senate Committee on GovernmentAl Affaire not later than 60 days 
after the date of the report. A"^ written statement must also be 
Qent to the House ajid' Senate Committees' oh Appropriations with 
the agency's first request for appropriations made more than 
60 days after the date of the report 

We are sending copies of this , report to the Director,^ Office 
of Management and Budget, and to the Chairmen, House Corftmitt^e, on 
Governijient Operations,' ^en^ite Coimnittee on .Governmental Affairs, 
House and Senate Committees on Appropriations, House and Senate 
Select Cojraaitteeis oh Intelligence, and House and Senate Commit-^ 
tees on Armed Services • ^ ' \ ' 



/ 



Sincerely yours. 





Cliftord^ 1'. Gould ^ 
Director " 
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DIG E'S T 

Th'e Defense Language Institute's Foreign Lan- \ ^ 
guage Center is responsible for providing , \. 
foreign language training to military person- 
nel who are being prepared for intelligence 
activities. The Institute's mission is to 
conduct and supervise language training for 
these personnel and .to provide technical sup- , 
port for alL-^ther foreign languag^e training 
conducted for the services except for mili- 
tary academies and overseas schools. GAO 
conducted this review at the req-uest of 
Congressman Leon E. Panetta, GAO's objec- * 
tives were to jLdentify and analyze signif- 
icant internal Defense Language Institute 
problems that diminish the overall effective-- 
ness of language training at the Institute. 

GAO reviewed the management of 'training at 
the Institute and concluded that changes are 
necessary to imprjove the quality of language 
instruction. More specifically, the Insti- 
tute needs to (1) replace outdated .materials 
in basic courses, (2) upgrade the management 
of classroom instruction, and (3) better as- 
sess the effectiveness of its training mission. 

EFFORTS TO DEVELOP 

QUALITY COURSE MATERIALS / i 
HAVE BEEN UNPRODUCTIVE ^ 

The Institute has made little progress in im- 
proving the quality of ^current course mate- 
rials. It has expended 159 staff-years at 
a'cost of about $4;2' million but has not pro- 
duced needed basic course materials. Defense 
officials said, however, that new course mate- 
rials would'be forthcoming during 1982. 

The lack of progresfs has been caused, in part, 
by the Institute's failure to (1) effectively 
set course development priorities, (2) properly 
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implement prescribed course development ' 
procedures, (3) adequately monitor progress of 
cciurse developm^ent; prpjects, (4) fully explore 
the potential of using commercial textbooks, 
and (5) effectiv^y use contracting to obtain 
needed course m^erials. (See p. 4.) 

CHANGES IN MANAGEMENT OF ' 
CLASSjlpOM INSTRUCTION SHOlJLD 
IMPRQVE TRAINING QUALITY 

Inadequate management of classroom instruction 
could be adversely affecting the quality of 
classrdom training. Specifically, GAO fodnd 
that: ' ^ 

--An officially approved and accepted teaching 
. methodblogy was lackirfg. 

--Instructor training was limited. . 

— Instructors were not being properly evalu- 
. ated by supervisors. 

— Response to an(3 followup on training recom- 
mendatioqs • were poor. , ^ 

— Technical language assistants had not b^en 
'•effectively used* (See p. 11.) 

ASSESSMENT OF TRAINING EFFECTIVENESS 
IS NOT CONCLUSIVE 

TKd- Institute cannot conclusively assess t;he 
prof iciency of its j^raduate^ or the effective- 
ness of, its t^-aining system on the basis of 
it;s existing evaluation processed • It needs 
to "forjnulkte a .cohesive policy statement on 
it'6 tri&ining objectives and standards in order 
to t^ain' students to desired user prof ic.iency 
level's. -The lack of clear objectives and 
standards caus-es confusion pver what to evalu- 
ate' or what'tfte proficiency level of graduates 
stiould be. .(See^ip. 17.) 

^ RECOMMENDATIONS . * . ' . 

GAO recommends that" the Secretary of the Army 
di>ecl the Commandantr of the Institute to: 

--De^/elop -resideftt basic^^ourses using commer- 
cially aviilable mate^als whenever these 
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<^an be adapted at less cost an<| in less 
ime than in-house development effort. 



-/Establish controls over cOkurse development 
projects which provide the means to assess 
progress against specified target dates. 

--Develop and distribute a standard schoolwide 
training^ methodology for use in all language 
departipents • • , ^ 

— Require all newly hired instructors to com- 
plex both phases of the basic instructor- 
training course. - In addition, instructors 
should*»'be encouraged to seek out additional 
training to improve their instructit^nal 
abilities as part of the individual' develop- 
ment programs. 

-^1 • 

— Esteiblish procedures to carry out the rein- 
stated policy for supervisory classroom 
visits and ^old supervisors accountable for 
routinely observing instructor' classroom 
behavio]^. 

— Establish realistic training objectives based 
on mission requirerpeht and use the Defense 
Language Proficiency Test to measure students* 
^performance In satisf acti6n of these objec- 
tives and require that sjtudents pas^ the test 
as a condition for graduation. 

— Develop a Defense Language Proficiency Test 
that will measure student proficiency of 
the objectives and standards established 
by the Institute. , ' 

AGENCY COMMENTS ' 

The Department of Defense commented that it 
sha>res a common interest with GAO in the eff-ec- 
tiveness of training at the Institute^ and statec?*' 
that several years ago it observed many of the ' 
conditions reported by GAO. Defense of ficis^l^ /^^, 
reported that/, since GAO*s audit ,ef forts werfe com 
pleted, there has been a high level of. accom# 
plishmeni: and that many problems noted in thi«' . 
report either have been or ar« b^ing addressed. 

GAO modifji-etK^some of the pAposed recommfehdat'ions 
ii%its draft\eport as a reslilt of Def^naN^^s 
comments regarding actions taken or underway. 



These modifications are addressed in the 
r.ecomnendation and agency comment sections 
of chapters 2, 3, ahd 4. (See pag«8 10, 15 
and 21. ) 
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CFxAPTER 1 

** — ■ > 

, INTROPUCTlON 

The Defense Language Institute (DLI), Foreign Language Center, 
in Monterey, California, was established to providej foreign lanr 
guage training for the entire Department of Defense (DOD). DLI 
is under the administrative control of th« Department of the Army 
and more specifically un^er that of the Army Training and Doctrine. 
Comn^^nd (TRADOC) at Fort Monroe, Virginia.- DLI ' s mission is very 
important because its, products, foreign-language-trained personnel, 
support the defer^se intelligence mission. Although linguists Con- 
stitute a very small^ percentage of total military pergsdnnel, DOD ^ 
considers them an essential ^element for preserving national 
security. ^ • ; 

DLI conducts full-time intejnsive foreign language training and 
provides technical cqptrol for all otJ;xer foreign language training 
conducted in DOD, except 6or milita|^ academied and overseas DOD- . 
operated schools. The instructional program is un^uely geared to 
the needs of defense, and most DLI students are active duty enlisted 
service members who eventually are assigned to defense intelligence 
jobs. DLI's basic resident courses, those taught at the Presidio 
of Monterey and at Lackland Air Force Base, Texas, ar-e 'aimed at 
developing working level competencies in listening comprehension, 
reading, speaking, and writing. DLI a^lso develops nohresid^nt lan-^ 
guage training programs for people in military field units and , 
elsewhere to .regain, maintain, or enhance language proficiency oy 
-jobs and missions. In addition, DLI is responsible for 

— developing and maintaining instructional material for both 
"^he resident and nonresident programs; 

— planning for faculty development; 

— empioying, training, and maintaining qualified subject 
matter experts in job and task analysis, testing, evalu- 
ation, curriculum development, ahid instruction in foreign 
languages; arK3 • ' 

» ^ ^* 

— exercising quality control over the foreign language 
program by providing standards and tests to measure 
language proficiency. 

13I?I currently provide? training in about 37 major languages 
and dialects; it relies almost solely on native-speaking instruc- 
tors. DLI exercises velry little real control over the number^ 
and timing of students scheduled for language training or the 
languages to be taught. U^er agencies determine terminal learn- 
ing objectives (required language skills) and, in conjunction 
with DLI,. establish the length of time students will be in train- 
ing. Except for the Army, the services also maintain adrtlinistra- 
tive control over their own students while a>: DLI. Coordination 



between DLI and user agencies is done primarily through an annual 
program review at the beginning of each calendar year. Staffing 
at DLI includes some 350 military personnel and a civilian work 
force of 850, of which about 6,00 are faculty members. -DLI teaches 
foreign languages to. about 3,500 service students per year -(the 
stodent load averages ab9ut -2 , 600 ) ; of which the largest number 
belongs to the Departmgint gf the Army. (App. II lists student 
enrollment and number of instructors by language as of Septentber 
198i. ) 

QUALITY OF LINGUISTS AND TRAINING f 
- HAS BEEN .A LONGSTANDING CONCERN 

The quality of foreign language training and' the competence 
■of military and civilian linguist^ have been longstanding con- 
cern^ For example, we -reported in 1973 1/ .that f oreign-language- 
•tVain-ing programs did not always give personnel the proficiency 
required to do thei^obs. We also reported in 1980 2/ that DOD 
had a large number language-'designa ted positions either un- 
filled or not filled at' the required proficiency level. Ii? addi- 
tion, what they learned was oft^n not s-pecif io*Aly related to 
-the requirements of their jobs. User agencies <pilso have hedome 
increasingly vocal about the inability of DLI-trained personnel 
newly assigned at duty stations to perform basic linguist duties. 

• In addition, DLI ' s own evalqatiops of tactical and strategic 
intelligence units, conducted in fiscal years 1979 an'd 1981, con- 
firmed users' cfomplaints. A major reason often cited by linguists 
and their supervisors for language deficiencies was inadequate 
basic language training wh.ile at DLI. 

OBJECTIVE,, SCOPE, AND METHODOLOGY 

At the request of Congressman Leon E.^Panetta, we reviewed 
the operations and training at DLL This review was performed 
in Monterey between January and September 1'981 in accordance 
with our Office's current "Standards for Audit of Governnrental 
Organizations, Programs, ^Activitie$, and Functions." Our objec- 
tive was' t5o identify and analyze those , significant interna 1 prob- 
lems that diminish the overall effectiveness of language training 



. Congressman Panetta agreed .thafwe would address\only those 
issues which related directly to DLI training capabili^s and 



l/^Need To Improve • Language T-r^ining* Programs aWd Assignments 
for U.S. Government Personnel Overseas" (B-176049, Jan. 22, 
1973) . • 

2/"More Competence "in Foreign Languages bTeeded by Federal Per- 
" sonn^l Working Overseas" (ID-80^31., Apr. 15, 1980). 
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were within the jurisdiction of the Commandant of DLI to correct. ^ 
The three issues examined were the adequacy of (1) course develops 
ment activities^. '(2) management of classroom instruction, and 
13) evaluation of /^raduates/ak<J^ traihing . 

^ /' /' ^ 
Our review included $in' analysis of previous studies (see 
app. Ill) 'of DLI and ^n examination of 'the fiscal years 1979 
and 1981 external field ^evaluations that DLI performed. We did 
not verify the- accuracy of findings reported by these eva^lua-- 
tigns, nor did 'we evaluate the met^dology used in^ making the 
evaluations or in developing the respective findings. ^ 

Our work -also included examinations of various internal dodu- 
'hients, such as DLI Instructional Systems Audits'; recently com- 
pleted student andifaculty questionnaires; DLI regulations and 
internal documented policy guidance ori instructional n^hodology, 
testing and grading, and course development activities and in- 
structors* training and appraisal records. We also reviewed stu- 
dent end--of-coursel»grades and compared them with students' language 
proficiency test results to determine jf students could attain the 
level of proficiency required by Users. 

' Interviews with department heads, supervisors, .instructors, 
and students were confined to the six largest language depart- 
ments-^-Rus^sian, German, tCorean, Chinese, Arabic, and Spanish— -which 
in totc^l account for over 90 percent of both faculty and^ students 
at DLI. Individuals interviewe^^ were randomly selected to obtai;i 
a cross section of opinions. Howev.er, thes« sf»lections do not -con- 
stitute a statistical sample and, therefore, opinions expressed do 
> not necessarily represent the views of all DLI Acuity and students 

Other internal problems at DLI, such as th^e questionis 'of com- 
petitive versus excepted service status for DLl faculty, employee 
morale and grievances, abolishment of the Academic Senate/ cross- 
cultural communication difficulties', and organizational structure 
iias^ues, were not reviewed, as agre'ed wittiSflongre'freman Panetta. 
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DLI'S COURSE DEVELOPMENT EFFORTS 

HAVE BEEN UNPRODUCTIVE 

. Students are not rece.iving uo-rto-date langiTage instruction 
DLI official^W user agencies' representatives r and others ac- 
kJiowledge tbaK^LI's resident courses are outdated, but DLI has 
made little pro^i?*as in developing new resident materials *f or 
basic courses. Between fiscal year 1978 and^he second quarter 
of fiscal year 1981, DLI used. about $4,2 million and 159 staff- 
years for course development and has yet to develop updated basic 
course materials, DLI*s lack of progress has been caused largely 
by' DLI* 8 failure to 

effectively set course development priorities between 

^its resir'ent and nonresident courses, 

—properly implement the Interservice Procedures for In- 
structional Systems Development promulgated . by TRADOC, 

^ — effectively monitor the progress of ongoing course de- 
velopment projects, 
* 

• fully pursue acquisition of commercial texts as an 

alternative t5 in-hpuse course development, and 

use contracting effectively to obtain needed materials 

anx3 to increase the use of- in-house resources. 

DLI officials acknowledged that the Directorate of Training 
Devefopment had not completely rewritten any resident basic 
courses. However, lack o£ progress was attributed to the (1) de- 
velopment of nonresident materials requested by user agencies* for 
worldwide use> (2) rigorous aad time-consuming requirements of 
TRADOC's Interservice Procedures .for Instructional Systems Develop- 
ment, .and (3) numerous delays in completing course development 
projects caused by project staffing dif f icXilties and 'interruptions . 

FAILURE TO EFFECTIVELY SET PRIORITY 
ON RESIDENT BASIC COURSES RESULTED IN 
DISPROPORTIONATE AMOUNT OF RESOURCES" 
SPENT ON NONRE SIDENT COURSES 

— : . f 

Despite the need for new resident bas-ic courses, DLI >has not 
effectively set priority on these projects and has spent dispropor- 
tionate resources on nonresident course development. 

DLI develops materials for both resident and nonresident 
courses. Resident training is that training which takes place at 
the Presidio of Monterey and Lackland Air Force Base and consists 



primarily of the basic, intermediate, and advanced courses. 
Monrfesident training, as the name implies, is designed for use 
at military aci^vities where linguists are employed. 

New resident basic course materials are ^ 
needed but have not been developed ^ 

'Many sources have noted that DLI ' s resident courses need 
revision. In J.979 it was reported that resident course mate- 
^ial s ranged in age from 4 to 27 years and desperately needed 
Attention. Despite DLI's c^burse development efforts, howevfer, 
no new resident courses have been implemented since 1976. Fur- 
thermore, DLI's primary user agepcy, the National Security 
Agency (NSA),- during a special program review conducted at DLI 
during December 1980, charged that: 

"While 177 manyears have been expended in course 
development pot a single resident course has been 
completed. " 

DLI officials do not dispute the fact th.at resident coM^rse mate- ^ 
rials are outdated. 

DLI has no system ""to effectively 
set course development priorities 

DLI's course development process depends upon obtaining the 
^cons^sus "of the user agencies during the annual program reviews. 
DLI officials told us there was no formal list of priorities; 
however, priorities are now stated wit>iin the Five-Year Plan for 
course development. User agency officials told us that, prior to 
the approved Five-Year Plan, DLl had been unable to set clear* 
course development priorities because there had been "o^f o^^^^^J^f . 
among the user agencies as to what courses should have the highest 
priority. For example, while NSA placed its priority on resident 
r-ourse development, two Army commands more concerned with 

* obt^iinwig nonresident materials. In addition, the Marine C«^s, ^ 
while it concurred in the need to place priority on development 
of resident basic ■ courstes, also desired further -development of 
Training Extension Courses. The Marine Corps later objected when 
DLI curtailed some extension courie development m favor of Dasic 
course projects. ■ * / 

Disproportionate amount of resources 
have been spent on nonresident- courses 

DLI has -not balanced the priority for its course developm.ent 
-needs. A disportionate amount of resources have been expended 
on nonresident courses in trying to satisfy the diverse needs of 



user agej^cies. 
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Our compQtatipns, made from data in DLI ' s . records , show that, 
of the 159.1' staff-years expended for course development between 
^scal yea-r 1978 and the second quarter of fiscal year 1981, o^^V 
33 -percent was* spent on resident courses while 67 percent was spent 
on nonresidenb-couraes, as shown by* the following table. 

Resident "'courses: 

Basic course development 

Basic course, revision * 
Intelwediata/and advanced course development 

1. Total 




52.6 



* Ncnresidwt courses: 

. Hea^start course development and revision i^./ 

Gat6\<^y course development and revision 1^*2 

^ Refr^esher and maintenance course development 13.2 

Traininq extension course development 60.5 

VTotai/-; ■ ^ . hBM 

DLI H AS IMPROPERLY IMPLEMENTED THE 
INSTRUCTIONAL SYSTEMS DEVELOPMENT APPROACH 

DLI -use* TRADOC'-s instructional systems development approach 
quidance'fox language. couSrse development. DLI has adcjpted the 
approach because it contends that it is the best method for de- 
veloping training that effectively meets user needs. H°^e^«f' 
found tha^ 'DLI-had improperly implemented this approach for some 
■of its higb»-d%nsity basic bourses now 'undergoing revision. 

<DLI Memorandum 5-2, "Planning and Management of Training 
Development;?rojects," dated March 15, 1979, P^°^i^^f ^^^fl^, ^.^^ 
training development be accomplished in accordance with TRADOC 
Pamphlet 350-30, " Interservice Procedures for Instructional Sys- 
tems DeveloRment." The process, as Retailed by Pamphlet 350-30, 
outlines fiVfe sequential phases in tl»e development of training 
materials': - analysis, design, development, implementation, and 
control. found, however, that for at least three courses be- 

ing revised— Basic Russian, Basic. Chinese Mandarin, and Spanish 
refresher/maintenance^-DLI had conducted the P^^«^%^ " 
order. In all cas.es, ^the development phase preceded the analysis 

phase. H 

Lack of appropriate front-end analysis before designing and 
developing'^courses, has also been cited in a previo«^ external 
evaluation as a roadblock to successful course development. A. 
1979 TRADO'C Inspector General evaluation noted that no signiri 
cant imprqvements had.been made in the basic resident course 
since TRADOC's prior 1978 annual inspection. According to the 
evalii'ation, the primary problem hindering effective basic 
course fcev;iopment was the lack of analysis of the basic language 



tequirementa to determine wtiat skills should be taught. The 
Inspector deneral added that "DLI's Analysis Division lacked 
guidance ^>n establishV*? priori-ties, in o^rdef to best use ex- • 
tremjely limited -resources. 

A Training Development official said DLI had deviated from 
"the instructional systems model because during 1976 considerable 
emphasis by the Commarfdanf was placed on the need to update old 
coursfe materials. Consequently, in trying to expedite develop- 
ment of new materials, Trai«ing" Development gave less attention 
to analysis and design while prematurely fpcusing on development. 

DLI HAS NOT ESTABLISHED ADEQUATE 'CONTROLS 
BY WHICH TO MONITOR AND MANAGE PROJECTS 

. DLI has not established adequate controls for monitoring 
progress and managing the development, of course materials. Proj- 
ect work' plans arfi constantly being revised to reflect the cur- . 

'rent "situe^tion; project status teports contain inaccurate data? 

•artd the lack of a stahdard for measuring productivity has hindered 
DLI's ability to monitor and manage course developnent , 

> As a management" tool the work pl^^ns are of limited use be- 
cause of constant revisions. DLI has allowed the project teams 
to revise their work plans to reflect current estimates , thereby 
lihiitin^ their value as a baselin,^ from which to measure vari- 
ances, assess the reasons for variances, and majje needed correc- 
tions. We were ttSld that DLI had not required teams to conform 
with realistic work plans because the project officers were re- • 
luQtaRt to commit themselves to milestones. In addition, offi- 
cials stated that/resource and staffing priorities were so 'er-. 
ratic that realistic planning was meaningless. Because of the 
absence of records, we could not determine the amount of slippage 
the orig^al work plans had undergone. 

Project status reports aire another management tool. Accord-*- f 
ing "to DLI Memo 5-2, these reports should establish and maintain ) 
continuous records on cost, time, manpower use, work accomplish- 
ments, and developmental problems. They should also .help man- 
agers to (1) project future developmental resource requirements, 
(2) iBat:h. make or buy decisions, and (3) perform problem-solving 
analyses. However, the reports, cannot measure the progress of 
development .because of changing work plans as discussed above, 
nor do they accurately report the staff-hours charged to projects. 
In a sampler of 11 of the 20 projedts ongoing, dbring the second 
quarter of fiscal "year 1981, the staff-hours charged for each pro- 
ject on the reports did not agree with those on 'DLI's computer 
system". -The discrepancy ranged from 21.9 to 180.9 staff-hours. 

■ ■ * S ' 

In addition, DLI harf^not developed or used performance stand- 

ar<^s to'm^sure the productivity of its project teams. NBA uSfeft, 

a 6:1 ratio? that is, the number of st^ff-hours required to. develop 
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materials 'for 1 hour of classroom infit ruction,^ -^s a standard for . 
developing its language courses. While DLI ^argued that tjlSA's 
ratio was unrealistically low, DLI still has not seriously tried 
to develop its own standards. It has be*en s^uggested that DLI ob- . 
tain additional staff to develop standards; however, action on this 
suggestion -is pending the results of a planned staffing survey. ^ 

DLI HAS NOT FULLY EXPLORED THE POTENTIAL ^* * ; • 

OF USING COMMERCIAL^ TEXTBOOKS ^ ' - 

Usage of Gonunerciai texts has been minimal despite DLI*s 
policy requiring .such materials to be evaluated and used whenever 
justified on a cost, quality, or timing basis. We could not find, 
ncr could officials provide records t« indicate, that J)LI ^^ad 
formally evaluated or incorporated commercial texts before initi- 
ating costly and time-consuming In-h^use development. We were 
told that, althouqli project teams reviewed commercial texts, Dt-I 
had not documented the evaluation procefes, nor had it provided 
specific guidance to the teams on the content,^ methodology, or 
extent of the evaluations. 

For example^, "Deutsch activ," a German textbook, was re- 
viewed by DLI staff and was said to be excellent for its superior 
use of communicative skills. How4^er, a formal evaluation Com- 
paring the text to DLI needs arid a quantitative analysis of what 
it. would cost to adapt and use the textbook at DLI we r^ never 
performed.- Regardless,- DLI awarded a contract for $25,460 for 
initial development of the German Basic Course. The contract 
was* not successful, and DLI is now trying to develop the German 
Basic Course in-house using -portions of the "Deutsch activ" text, 
pending an agreement with the German publisher. 

DLI staff have raised several objections to using commercial 
texts. We werfe told that commercial texts were geared to a differ 
ent audience, they lacked -military "flavor" or te«rminology , or 
copyright and* availability problems would interfere. These objec- 
tions, however, have been discounted by user agencies and other 
DLI staff for the »f ollowing reasons: 

— A g6od basic text could serve as the. framework for a course 
with. additional exercises and otiher supplemental materials 
to provide tlxe intensity needed by DLI. ^ 

— Basic courses are not military specific until the end, andv 
, ^military terminology could easily be added. 

— NSA and the Foreign Service Institute (FSI) both use c<am- 
mercial texts extensively for their language courses. 
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dli has had poor results in contracting 
' pgr basic course development 

DLI'S efforts to contract out .basic course development have 
not been successful. Officials acknowledge that contracts for 
course development .between 19t9 and 1975^roduced little usable 
materials, and no completed basic courses were ever^elivered or 
put into use at DLL The only contract for basic course devel- 
opment since this period did not produce satisfactory results 
either. Failure of the 1969-75 contracts has been attributed 
primarily to poor contract specifications. 

- * DLI has entered into only one contract for basic course 
development since the 1969-75 perio.d; this was for the German 
Basic Course. The contract wa^ ayarded in September 1980 for the 
amount of '$25^460 and was terminated in May 1981. Although all 
the lessons specified in the initial contract were received, the 
materials were not usable. According to DLI officials,, specifi- 
cations were nof. at fault for failure of the contract. Instead, 
they said, DLI ' s inability to .effectively monitor and control 
the contract caused its failure. 

CONCLUSIONS ^ 

Existing basic -courses have deteriorated ''tg the point where 
there is a detrimental effect <,n the quality of training. Course 
• development projects have proceeded slowly because of .^^e la^ of , 
appropriate priorities, improperly implemented "course development 
procedures, ineffective monitoring and management of development 
Sirojeqts, failure to fuily expire the potential benefits of 
commercial texts, and unsuccessful corftractmg efforts. 

DLI -needs to develop a system brfsed on i'^ternal as well as - / 
external inputs' for assessing course development ptitority needs. 
. Establishing controls over its course development P^°Jf^^^-%^^%^^ , 
.with an evaluation of alternative course materials Should, improve 

course development. 

RECOMMENDATIONS ~ ; \ ^ 

We recommend that the Secretary .of the Army direct, the Com- 
mandantaof DLI . 

, —develop resident b^ic courses using ^o^^^^^^^i ^rrolt 

able materials whe3lver th;ese can be adapted at less cost ^ 
and in less time than, in-house development and 

^ -establish controls over course develo^n^ P^°^ects which 
provide the means to assess progress agal«^ specitiea 
target dates . ' ■ 
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AGENCY. COMMENTS ^ • • ^ 

t 

/in thfeir March 19, 1982, coinments (see app.^I), DOD officials 
agre^^'tiha^^I couid have iSetter managed its. course development 
resQurces; however, they emphasized that their investment in 
.course development would begin -to be realized in 1982,with the 
cbmpletion of the Basic Russian course. We have not Verified 
that DOD will meet the projected completion date for the Basic 
.Russian course. In addition, DOD in March 1982- reported the sta- 
tus foir several additional ba-sic language courses but did not 
provide any ^estimated completion dates for these courses, and we 
have not attempted to verify the' provided information. While DOD 
commented that it had used and adapted commercial materials for 
several of its courses, ,we found only very limited use of commer- 
cial material^ and continue to believe that gl?eater use is neces- 
sary if the Institute is to achieve its course development goals. 
DOD officials reported that production control measures h^d been 
recently instituted to more closely monitor the progress of course 
development activities. 

DOD comments indicated that the 5-year training development 
plan establishes project priofitie?, and the Institute and user 
agencies now agree on resident and nonresident course development 
priprities. DOD specifically commented that, as of March 1982, 
74 percent of course development resour-ces have been allocated for 
resident prggrams while 26 percent have be4n allocated for deyel- 
ppment of norfresident and refresher/maintenance program^. Accord- 
ing-ly, we have dropped our proposed recominendation*To establish 
a more, effective process for setting project priorities. 

DOD commented that it had accomplished our proposed recommel^- 
dation to .establish controls over course development activities 
by late 1981. However, the recency of DOD*s actions and^the lack 
of information as to how these actions will achieve the intent of 
oqr proposed recommendation cause us to continue to believe that 
controls are nee^d« 
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CHAPTER 3 . 

CLASSROOM INSTRUCTION , C'^ j^E .BETTER 

MANAGED AND gUPtlkVISED ' 

Important elements of classroom instruction could be better 
•managed- and supervised. Many ptoblems have fee^n^^^^^^^^ii^^i" 
past studies conducted by DLI and- other organizations. However, 
to date, some important training policies and procedures are 
either lacking or unclear or have been poorly intplemented . we 
.found thatt ' ' 

—DLI lacked an officially approved and accepted teaching 
methodology for instructors to use. ■ • 

— I^tructors, once hireta, received only limited training, 
^^Ir classroom instruction. ^ - 

—Instructors we r^ not being adequately evaluated on their^ 
'. instructional capabilities. 

— DLI's evaluations of training quality were not effe^ve 
because of poor response to recommendations and ina<^uate , 
foilowup on them. , ' . 

* —Technical language assistants (TLAs)^ provided to DLI have 
not been used effectively. - . ' ^ 

While thes^problems have not been solved, re<mnt .DLI initiatives, 
such as revising inst rue tOB^hi ring stiindards, instituting a new 
brogram to improve the" ua/of TLAs, and creating and fi 11^1^9 ^he 
position of Academic DeafT, are all .aimed at impr^ing clafiTsroom 
instruction. * * 

OFFICIALLY APPROVED AMBrt^CCEPTED 1 
REACHING METHODOLOGY IS UNKING . * 

DLI has/had no definitive policy on methodolo^ or the theory 
of foreign language traifting since January 1976. -Before 1976, a 
'definitive "Official pblioy" on methodology was contained in.DLI V 

• Pamphlet 350-4, entitled "DLI Guidelines." This pamphlet dealt . 
with the principles and methods of teaching and learning in the 
Defense Language Progrhm, However, in January 1976 the pamphlet 
was rescinJ^ed and has not been replaced; Although two memorandums 
dealing w^th course methodology were written after the pamphlet 
was rescinded, they were not adopted Internally by DLI as official 

I guidanafe to the departments. ^ i 
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Teaching methodology at DLI varies widely even within the 
same language department. Interviews with department heads, ^ 
supervisors, instructors, and students substantiated the use 
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of different methodologies. They variously described the 
methodologies as "eclectic, audio-lingual, cognitive, inductive, 
pluralistic, improvisationa^i, and doing their own thing." One 

H instructor claimed that six different methodologies were used • 
indiscriminately in his department. Othe^- instructors said they 
had no official DLI methodology, or they simply followed the text- 

, bjok. DLI officials acknowledged that the use of various method- 
ologies had an unpredictable effect on the quality of instruction 
and that language departments had, in effect, been allowed €o do 
"their own thing." They also acknowledged that DLI needed to de- 
velop and "package" a methodolpgy to make it easier to understand 
and follow. "5 ' ' 

INSTRUCTOR TRAINING IS LIMITED 

Many of the newly hired instructors a^e not cfmpleting basic 
instructor training', and eve*n fewer receive additional training 
for self-development ajid job ^vanceipent. ^ Although DLI gives most 
new instructors some training and orientation, it has not been con- 
sistent in routinely updatin/g and reinf orc-itig the earlier training 
with additional* tra|.ning. 

■draining records showed tSiat 77 percent of instructors hixed 
between January 1980 and May 1981 received "Basic Instructor 
Training, Phase I." ThiS ■2-week course, supplemented by a 1-week 
in-class observation, is .designed to gi-ve native or near-native 
speakers of foreign languages^„the akills/ knowledge, and abili- 
ties to function as DLI instructors. "Basic Ini^tructor Training, 
•Phase II," is an observation period during which an instructor is 
evaluated on how well he or she applies the techni(pes learned 
in phase I. During the same peuiod, however, only 16 percent of 
those who compl0Le6 phase I competed phase II. Further, • records 
indicated that, djaring this p€ri«l, few instructors attended other 
DLI Courses. ' . ' ' . 

SUPERVISORS NOT PROPERlV EVALUATING 
INSTRUCTORS [ ] 

L^ngua^e department supervisors are not properly evaluating 
instructors' classroom perforfliance . DLI guidelines ^ecify the 
most' important tool ih perf ortttance eval,uation is the supervisor 
audit. This is an ufischeduled visit to a class by a department 
supervisor for observing and ^recording on-the-job behavior and 
appraising performance. Each, audit should include such events 
as observing behavior, wtitin.g observations, discussing perform- 
ance with the instructor, counseling the Instructor as required, 
and insuring the appropriate observation form is cosigned by the 
instructorBk ' i> 

The supervisor is responsible for observing instructor- 
performance for a full' teachihg period at a minimum of 6 times 

■ n ^ 
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a year or more .often if necessary. Instructor olbservatibn forms^ 
serve as records pf^an instructor's performance and as support 
' for written performance appraisals. 

While, in theory, the supervisor visit *is an important evalu- 
ation tool, we found that supervisgrs were not carrying out ±hese 
responsibilities. Fox -example, we interviewed 14 of th^ 28 super- 
visors (or 50 percent; in the 6 largest departments, al| of whom 
indicated that' they did not visit their instructors' classes reg- 
ularly; about half indicated they visited classes as infrequently 
• as twice a year. Furthermore, they difl not always record their 
observations* Also, instructors in four of these department's said 
they had not been pDunseled by supervisors, although this is re- 
quired after the visit. . ' ' ' • , 

DLI officials said they were developing a new performance 
appraisal policy and related procedures which were expected to 
provide additional guidance on i^structoif evaluations. 



RESPONSE TO TRAINING RECOMMENDATIONS 
HAS BEEN POOR AND FOLLOWUP HAS BEEN 
INEFFECTIVE 



* Reviews of DLI's instructional delivery systein. Instructional' 

Systems Audits (ISAs), are performed to determine the means of 
improving the effectiveness or efficiency of classroom instruction. 
However, procedures for monitoring and implementing ISA recommen- 
dations^were not followed.. Specifically, we found that: 

— The Directorate of Evaluation had never implemented moni- 
toring procedures for ISA recommendations although a 
September 1, 1978, DLI memorandum indicated such procedures 
existed. , • 

A ■ 

— The Directorate of Training had delegated to^its individual 
language departments the responsibility for implementing 
ISA recommendations and had not insured compliance. 

The Director of Trainii:ig acknowledged that ifnplementing ISA 
recommendations had ^3een left to the departments. The Director 
^Baid he had made a conscious attempt to decentralize authority 
and thereby allowed the departments greater control.. 

This delegation of authority, however, apparently did not 
result in timely training improvements. In a memorandum to the 
Director 'of Training in September 1980, the forme r Commandant at 
DLI noted that one department's reply was so general that ft led 
him to believe the department took the ISA report under advise-- 
ment rather than identify the tasks needed t6 bie accomplished in 4 
ordef to enhance the operation of the department and that, had a 
status report not been requested, the recommendations would not - 
have^been seriously followed up . ^ 

13 ' ^ ^ 
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Iti an apparent effort to improve department responsiveness ^ 
to ISA recommendations, a revision to DLI MenK) 350-5, dated 
January 15, 1981, assigned monitoring of ISA* recommendations 
to the newly formed Office of the Academic Dean.^ However, to 
date, written procedures for carrying out this responsibility 
have not been developed, DLI officials told us, though, that, 
as of September 1981, new ISA procedures were being readied 
for dissemination. 

TLXs HAVE NOT BEEN USED EFFECTIVELY ^ 

TLAs- have not been effectively ilsed, and no central afuthority 
has coordinated their use by individual departments. About 60 
TLAs have been assigned to language departments and course develop- 
ment. These were career military linguists with field experience 
who could give students a practical view of the application of 
foreign language training to actual job duties. Their duties at 
DLI inbluded, but were not limited to: 

— Explaining military terminology. 

— Assisting in conducting and grading language laboratory 
work ... , ^ 

— Assisting faculty in classroom instruction, administering . 
tests, r supervising study halls, and tutoring. 

During the Special Program I^eview in December 1980 and the 
Annual Program Review in February 1981, "disagreement arose be- 
tween DLI and NSA regarding the TLAs' role. NSA contended that 
DLI intended to use the TLAs as counselors rather 'than to assist 
in mission accomplishment as originally intended. It further 
contended that tfcis shift had a "deleterious effect upon both 
TLA morale and ef f ectiveness " and t^^ the TLAs * language exper- 
tise could be better j^sed elsewhere in the defense community. 
Although several of the depar^troents we reviewed had ass?.gned mean- 
ingful duties to TLAs, others had not. One department considered 
the TLAs to be "spies" for the user agencies and refused to allow 
'them significant roles in the instructional program. 

DLI officials acknowledged that some departments did not use 
TLAs Effectively. How«:yer, they "pointe*^ out that, in response to 
user agenciL criticisms and as an attempt to correct shortcomings 
in the previous memorandum of understanding', a new program had re- 
cently been established. This program, implemented on August 28, 
1981, is intended to insure proper use of the TLAs at. DLI as we/1 
as to develop .a "cadre of expert, military linguists." Ov cralK 
the program assigns to the Commandant operational control over all 
TLAs and assigns program responsibility to the Director of Training. 
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CONCLUSIONS ' 



DLI officiaTs must exercise greater oversight over the 



school's instructional system so as to insure an optimum level 
of instructional quality. Lack of official policy guijaance on 
trafining methodology, instructor training, and instructor 
evaluations and inadequate or untimely response to suggested 
--^^<1H>rovement8 to the instructional system are degrading the qual- 
ity of language training linguists receive- and could adversely 
affect their job performance. 

RECbMMENDAT IOInS - - ^ 

We recommend that the Secretary of the Army require the Com- 
mandant to: 

— Develop and distribute a standard schoolwide training 
methodology for use in all DLI ' s language departments. 

« * . > 

— Require all newl^i hired instructors to complete 'boiih phases 
of the basic instr\ictor-training course*. Instructors should 
be encouraged to seek out additional training to improve 
their instructional abilities as part of the individual de-- 
^ velopment programs. 

— Establish procedures to carry out the reinstated policy for 
supervisory classroom visits and hold supervisors accountalfPl 
for routinely observing •instructor classroom behavior. 

AGENCY COMMENTS 

DOD did nafe- directly address our recommendations to develop 
and distribute a standard training methodology for use in all 
language departments. It responded that DLI had begun a major 
revision to its entire faculty professional development program. 
DOD reported thit, from January 1980 to February 2?, 1982, a total 
of 506 faculty members received additional training aside from 
the Basic Instructor Training Workshops. As noted on page 12, we 
addressed only newly hired instructors through May 1981 and pri- 
• mari'ly the second phase o,f basic instructor training rather than 
additional training reported in DOD comments for its faculty. 

Revision of DLI faculty professional development program, 
realignment of the* Faculty and Staff Development Division under 
the Office of the Academic Dean, and changes to procedures for 
responding to the results of Instructional System Audits and 
field evaluations were reported as recent measures which should 
help to improve the management of DLI classroom training. 
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concerning -the proposed recommendation in our draft report 
for requiring additional training for new instructors, DOD offi- 
cials responded that the requirement for additional training for 
new instructors could b^st be incorporated in iAdividual develop- 
ment plans. We concurred in this approach for minagmg instructor 
traiBing and have modified the recdmmendation accordingly* \DOD s 
official comments are included as app. !•) , » 

The intent of our proposed recommendation on supervisory 
classroom vistts was.to estabUsh a policy and implement it. DOD 
commented that the policy fotf supervisory visits to classrooms 
which had been rescinded in the 1970s was recently reinstated. It 
did not comment on how the policy is to be implemented or if and 
how supervisors will be held accountable for following the policy. 
We have therefore revised our proposed recommendation to provide 
for a management control that can be used to insure compliance 
with the supervisory visits policy. 
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CHAPTER 4 . 

. , ^ DLI LACKS AN ADEQUATE MECHANISM FOR 

DETERMINING THE QUALITY OF ITS STUDENTS 

DLI's existing, evaluation process is inadequate for 
assessing student proficiency or determining how well the DLI 
is performing its language-training mission. Specifically, we 
found that: 

—DLI did not have a cohesive statement of its training 
objectives a.n6 standards. 

—Proficiency testing had not been adequately developed as 
an evaluation tool and the Defense Language Proficiency 
Test (DLPT) was not relied on for determining proficiency 
of graduates. ."^ ^ 

—Development of criterion-referenced tests, which measure 
achievemeht of users ' terminal learning pb^ective-s (TLOs), 
had not progressed. 

DLI D OES NOT HAVE A COHESIVE STATEMENT ' 
OF ITS -rt^AINING OBJECTIVES AND STANDARDS 

Sjefore any evaluation of DLI's training system can be effec- ' 
tive, there must be a clear understanding of exactly what the 
training objectives and standards are. Yet DLI 's objectives 
and standards- are not -clear, and this has caused confusion over 
what to evaluate or what th^ proficiency level of graduates 
should be. , . • , 

Since 1976 DLl'has not had a single cohesive policy docu- 
ment clearly explaining the trafhing objectives and standards 
and their interrelationship. Before this time, however, DLI s 
training ob:Nctlve was to give students a foundation in. the 
language sufficient to attain proficiency level 2 (limited worl^- 
inq proficiency) in reading and writing and level 3 (minimum^ 
professional proficiency) in listening and speaking, but such 
general language training was not designed to prepare sT:udents 
for any particular typ^g^f mission' or assignment. 

As described by fI[ ' s prof iciency index, one who has achieved 
level 2 for reading can r-ead simple narrative^ familiar subject 
matter and, aided by a dictionary, he/she can get the general 
sense of written comm.unications . ' One who has achieved level 2 for 
writing dan writdl sentences on familiar topics appropriately 
using techpical language vocabulary; errors in spelling and struc- 
txire- occasionally obscure the meaning of written m4terlal. on 
the other hand, level 3 listening and speaking rfequire more abil- 
ity. To achieve level 3 listening, an individual should be able 
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to under«tana general conversation or discussion on topics within 
his/her special field. Similarly, a person speaking at level 3 
must be aW.e to participate effectively ifl all general conversation 
and discuss particular interests in his/her special field without 
making errors that obscure meaning. (App. IV lists FSI's proficiency 
level descriptions.) i% 

In 1977, DLI requested and received from its user agencies 
a new se? of TLOs which more clearly specified the skills that 
students should master to be able to perform their language duties. 
The TLCfs consist of 25 objectives which, if achieved, would yield 
a basic language student able to assi;^e assigned linguistic; tasks. 
Examples of TLOs involve' such objectives as spoken interpretation, 
.reading interpretation, conversational response, transcription/ 
written response, etc. However, the TLOs when received did riot • 
specify quantitative "standards by which to measure achievement of. 
these objectives. Such standards should spell out the performance 
level that would be acceptable to users. For example, reading 
interpretation is one skill identified in the TLOs. To satis- 
factorily demonstrate attainment Of this skill, standards should 
specify the level of accuracy that would be generally acceptable; 
that is, the individual should accurately, interpret all information 
conveyed or 75 percent of the information and/o?jjiust be able to 
organize it in the sequence origini^f^ convey^. 

- Notwithstanding the lack of specific 8tanda?4f^» DLI, in 1977, 
began to incorporate the new TLOs into its policies and training 
program. However, without specific standardfrr^Ci did not know 
yhether the TLOs were being achieved, and DLI continued to provide 
instruction and graduate students on essentially the same basis 
that it had done before receiving the new TLOs. 

Because of the absence of specific standards, various DLI 
staff with whom we spoke had differing understandings of what. * 
DLI 'a mission and course objectives were* Some said DLI's^asic 
courses were supposed' to meet proficiency level 3,' other^told 
us level 2, and some said the courses should n\eet the TyJs • 
Notefbly^ there is no recognized connection between^ the prof i-^. 
ciency level descriptions and TLOs. 

The . Director of Evaluation, in a March 1981 memorandum to 
the Commandarit, ciharacterized the ambiguity over DLI * s mission 
objectives and training standards as a "systemic problem." He 
explained that TLOs were only tangentially addressed in the - 
course objectives, the graduation criteria, the instruction, 
or the final examination and that the ambiguity of , prof iciency 
level descriptions resulted in inconsistent interpretations by 
instructors* 

DLI officials with whom we discussed this matter, includ- 
ing the Director of Training Development and^ the Director of 
Evaluation, agreed that ambiguity irf mission statement and course 
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objectives and the lack of training standards were causing 
misunderstanding over what DLI should be e'FP^^i**^/^,^^^^";^^?;^^, ' ' 
aid that cohesive policy guidelines, were needed similar to those 
which existed in 197.6. 

CURRENT P ROFICIENCY TESTS ARE NOT RELIED 

ON AS A MEASURE OF THE EFFECTIVENESS OF , - ' 

JDLl's TRAINING SYSTEM ' ' 

The only yardstick for measuring the overall effectiveness of 
DLI training o^ st?dent proflcis^is the DLPT. Yeti even though 
DLI administers„theJ2I.ElPi^''trnot a graduation ^^^^33 
it is^nott relied^iSTas a primary measure of training eff^^^^^J^^" 
DLI offilials pointed out that the DLPT had been Resigned to sc teen 
personnel for general language proficiency .and had never been v^li 
dated agairist objective standards of proficiency. Nonetheless, a 
1973 GAO report 1/ concluded that reliable proficiency testing . 
was needed to evaluate training effectiveness. The DLPT admin- 
istered by DLI consists of 120 multiple-choice items cbvering^ 
2 (listening and reading) of the 4 language skills ^aujht at 
DLI (additionally, speaking and writing) and^takes about 1 hour 
to administer. Scores arfe converted into proficiency levels 
ranging from level 0 (no proficiency) to level 3 (minimuip pro- ^ , • 
fessional) on a scale developed years ago by FSI. 

Although ^LI administers the DL^T at the time of graduation, 
it does not relV on the scores -because there is no assurance of 
iow accurately proficiency, as indicated by the FSI proficiency 
levels, equates^o the DLPT because the method of convertij,ng or 
transforming DLPT scores, to FSI's proficiency levels ^as never 
beSn validated. -In other words, it has never been scientifically.* 
established as to- what raw scqres ftfom the DLPT equate to FSI 
levels. DLI officials acknowledged that lack of validation 
diminished confidence in the meaning of the proficiency levels 
assigned wh«>4|udent8 completed the DLPT. They said, however, 
that a project was underway which would establish generally . 
accepted standards so that DLPT scores would have more meaning. . 
DLI -8 Director Of Evaluation and the Chief of Test Division bot^ ^ 
agreed that" the results of this project, if successful, would be. 
a more useful measure- of proficiency and that it would not be 
unreasonable to require that students attain the J^^^^^ • 
in whatever training objectives that DLI decided, to establish. 

Students of DLI- graduate solely on the basis o^ scores they 
achieve on various interim tests and a final test at the end of 
the course, instead of proficiency test results. These tests 
measure achievement of ccSurse contents and are not related to 

" • 

l/"Nee<3 TO Improve Language Training Programs and Assignments for 
- U.S Government Personnel Overseas" lB-176049. Jan. 22, 1973). 
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the FSI proficiency levels. Although a correlation exists between 
•end-of-course grades and DLPT proficiency levels, DLI does graduate 
students vt^o.io not attain a high level of prof ici^ency even though 
some students achieve high scores for their final course grades.. 
For example, of abcuit' 26,000 students who graduated between 1974 
and 1981, -S#661 gr^^tes attained level 1 or lower in the listen- 
ing skill. Similarly, 2,354 graduates attained level 1 or lower 
for the reading skill. Due to the number of graduates involved, 
we di'd no^- try to determine why they could not attain level 2 or 
higher ^or these skills,. 'For example, we do not know the extent 
to which this miy be due to errors in measurement or clerical 
recording "errors and, thus, cannot say with any assurance whether 
students "have mastered language training at DLI. 

< •' ' 
. CRITERIONt REFERENCED TESTS HAVE 
^ BEEN DEVELOPED SLOWLY— TESTS" SHOULD ^ 
' REFERENCE TLOs 

— ■ ^ 1 *" 

' ' * t ... 

DLI's bourse tests have been the subject of criticism. In its 
review of- DLI graduates assigned to military field units, DLI's 
Directorate 'of Evaluation concluded that DLI needed to devise tests 
for measurirrg the skills students had been taught at Monterey., DLI 
. of ficiala-. acknowledged that developing criterion-referenced tests 
would overcome the deficiencies in existing tests; however, develop 
ment of these \ests has been ongoing since 1978, with very little 
progress, v " ^ 
» 

Criterion-referenced tests are designed to measure how well 
students 'hav^ learned language skills specified in tti^ TLOs. We 
agree that theae tests should provide a more objective measure- 
ment of ^achievement . However, we were told that these tests were 
not being written and validated directly to TLOs,. DLI officials^ 
said' theV-thad experienced* difficulty in developing these tests 
because TLOs *did not detail the level of Achievement needed. 
^ Therefor^, tests could ncrt measure how well course objectives, 
based on TLOsV had been"* achieved. Furthermore, DLI official^ 
stated that*NSA had declined a request to validate ULI t%fts in 
actual job environments. , 

Althpui^'DLI labete the tests it is developing as criterion 
referenced, we- were told that DLI had curtailefl its attempts to 
*write Jtest^ based on TLOs. Instead, we were told that the new 
tests being d^^^eioped merely represented^^chievement tests on new . 
basic course m^i'terials being developed. DLI officials claimed 
that, since new^courses were being developed on vthe^basis of NSA's 
TLOs^ nefw tesjts" would be abetter indicator of whether students 
had been 'able, to achieve stated objectives. 
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CONCLUSIONS 

Laclc .of clear and cohesive training objectives and -standards 
to measure language proficiency has prevented DLI from effectively ' 
appraising its training mission. Although we believe that student"? 
language proficiency is the best indicator for determij;iing the 
effectiveness of training, DLI * continues to appraise its overall 
training effectiveness on the basis of student grades and achieve-^ 

raent testQ. 

% r . 

* ifi line with our 1973 report, we continue to believe that 
valid, reliable proficiency testing is a key element of sound 
evaluation. Such tests would reveal whi(?h students were well 
.qualified for graduation ^ind could identify areas where training 
could be improved. Althotlgh DLI is developing new tests tp com- 
plement new cQurses, they may, not be fully satisfactory for 
determinirig the quality of its training or the skills of its 
gradirtTtes^ 

RECOMMENDAT IONS 

We recommend that the Secretary of the Army direct the 
Commandant of DLI to: ^ 

—Establish realistic training objectives based on mission 
requirements and. use the DLPT to measure graduate students' 
performance in satisfaction of these objectives. and require 
that students pass the DLPT as a condition for graduatiojfi. 

— Develop a DLPT that v/ill measure student proficiency of the 
objectives and standards established by DLI. 

AGENCY COMMENTS • . m ' \ ... 

.DOD agrees that establishing training objectives based on mis- 
sion requirements is the foundation needed before language profi- 
ciency can be measured. DOD reported that DLI is an active partic- 
ipant of the Interagency Language Roundtable which is reviewing , 
U.S. Government language proficiency start^ards. These standards, 
when published, will be the benchmarks for DLI training objectives. 

Furthermore, DOD agreed with our proposed recommendation to 
establish realistic training objectives which can be e^^aluated by 
using the DLP'T. At DOD's suggestioh, we modified our proposed . . 
recommenffetion to require the Secretary of the Army to establish 
realistic training objectives based on mission requirements and 
to use. the DLPT to measure performance . ^In addition, in accordance 
with DOD '6 suggestion, we added as part of this recommendation that 
passing the DL^ be a requirement for graduation and deleted the 
separate recCmmenSation which requested that the DLPT be used as . 
a requir€?men'€'for graduation. 
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DOD official* did not agree with our proposed recommendation 
to resolve the prbblem -of converting DLPT scores to FSI proficiency 
dSs^IptiSns bu? stated that DLI would redesign the DLPT to better: 
meet its needs for assessing student proficiency. We concur in 
• DOD'e approach to assessing student prpficiency and have changed 
our proposed recommendation accordingly. (See app. I for DOD s 
comments . } 
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1& MAR 1982- 



Mr. Clifford I, Gould 
Director, Federal Personnel 

and Compensation Division 
United States General Accounting ' 

Office 
Washington, D. C. 20548 

, Dear Mr. Gould: 

• 

This is in reply to y^r letter to the Secretary of Defens^e regarding your 
report dated February 18, 1982 on **Wea*r»esses in the Resident Language 
Training System of Defense Language Institute Affect the Quality of Trained 
Linguists'* OSO Case #5904, GAOlCflde 961149. As enunciated in the March 5, 
1982 meetir>g between Messrs. Kr^r and Esposito frtM GAO, and personnel 
from the Department of Army Staff and the Office of the Secretary of Defense, 
we share a common Interest in the effectiveness of training at the DLIFLC. 
We also feel that there has been a* high level of accomplishment at the DLI, 
and that many problems identified in the past, actions taken, programs 
developed, apd resources committed are oroducing desirable results*. Our 
investment in the DLI, like any other investment strategy, includes a 
commitment of resources—fUnds, .personnel , and management attention. Bi/t 
it also includes time, time for the investment to realize its dividend. 
Many of the obsefrvations of 'the GAO review team wejrfe also made by DoD • 
personnel, some, tf^ears ago. The effects of many 4f our corrective actions 
have, in fact, b«n realized since the departure of tht on-site GAO team 
last September (1981). Therefore, we think it appropriate to offer -as detailed 
an upda^as possible to the draft report in order for the GAO to present the 
most, accurate picture of this important program to the Congres^. 

As agreed to during the March 5th meeting, our comments are divided into two 
major parts and are attached: 

- Enclosure 1: proposed additional paragraph to Cover Summary; comments 
on Digest, recommendations, and proposed ^Agency Coftments" paragraph 

- Enclosure 2: a by-chapter update. 

We hope these comments can be usecf to enhance the report, and to picture the 
DLI in light of current efforts. Action in response to final GAO recommenda- 
tions will be accomplished by the Officfi of the Secretary of Defense and the 
Secretary of the Army as appropriate. 
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Recommended additional paragraph to 
. "C over Summary" 

GAO acknowledges that many of the. issues and problems cited in this study 
have also been recognized by the Department of Defense, and that corrective 
action has been started and in some cases accomplished. The- increase in resources 
cownitted to support the InstitiTte, the level^ of management attention devoted by 
Army and DoD-wide users, and recent internil organizational and leadership changes 
are indicative of movement toward an improved training effort. 

Comments on "DIGEST" 

P age i : (1) The DLI has no responsibility for foreign language training at the 
Service Academies or Mn^verseas Dependent Schools. >^ 

* (^Although changes in the "management of training" at^he DLI may 
help to "improve the quality of language trained personnel,' it 
is necessarvxto note that the training enterprise is only one 
subset of a much larger group of human resource considerations >^ 
affecting^ personnel quality-fo incltide a broad range of personnel 
administrative concerns, manpower management, compensation, 
recruitable"^ labor sources, uniqueness of mission,' def^loyment, ^ 
. and utilization. ' Undue weight qr\ management changes at DLI to 

change the character of Defense'human resources may be misleading. 
The DoD is. In fact, attempting to improve 1 inguist 'personnel by 
addressing a much brolder set of manpower and personnel issues. 

ii : (1) The investment of resources (dollars and manyear$) for training 
development has not been without yield. The investment started 
less than four years ago and new courses will be completed 
beginning calendar 1982 with continuous completions each year 
thereafter. A production time Is a necessary and understandable 
component of any Investment/development program. An accurate 
accounting of curriculum developments is offered in the comments 
on Chapter 2. 

(2) Lack of rapid progress' in course develotxnent is also attributable 
to a deliberate management decision to use limited resources to 
meet the priority mission— resident training of a rapidly 
increasing student lo^id. 

Comments on "Recommendations" 

Recommendation 13 - In addition to commercial sources, DLI has< also used other 
government agency and university programs. ^ - 

Recommendation #3 - Already accomplished 
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Recommendationi5 -'Recommend second sentence to read, "... to seek out a 

additional training to improve instructional capabilities as part of . 

Individual. Development Programs." 

Recommendat'ion #6 - A policy of supervisory visits to classrooms exists.. It 
^ had been temporarily recinded but has been reinstated. 

Reconmendation #7 -.Training objectives are not derived through use of proficiency 
tests. Recoitsn^mi rewrite to read: 

"Establish realistic training objectives based on mission 
requirements. Use the Defense Language Proficiency Test to 
measure 'student performance in satisfaction of those objectives., 

Reconmend ation #8 - Nonconcur. DoD has no intention of converting OLPT scores . 

to FSr proficiency levels. Current work in redesigning DLPT s is being 

coordinated with fche State Department and other interes^^d agencies. 

Reconmendation #9 --Recommend mer^e with recommendation #7, they are almost 
~ the same thing. 

Recommended Additional Paragraph 

Agency Coiranents: 

T^^'e Department of Defense shares a common interest with the GAO in the 
effectiveness of training at the DLIFLC. We feel that there has been a high 
level of accomplishment and that many problems identified in the past, actions 
taken programs developed, and resources committed are producing desirable, 
results The increased investment in the DL'I should be viewed as any other 
investment- resources plus time yields a dividend. Many of the observations 
of the GAO review team were also made by OoD personnel as long as several years 
ago. The impact of njany o'f our corrective actions is just now being felt, 
even though some were instituted years ago. 

Pie Defense Foreign language Training Program, and the Institute in ■ 
Monterey are high priorities for the Department of .the Army and the entire 
Defense community. We believe it is well on its way to accomplish missjo/i 
requirements in a most effective and efficient way. / 
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The following conwents are offered to. provide a more complete picture of DLIfLC 
operations. Some of these comments refer to past actions, some to recent DLIFLC. 
in-house initatives, and some to new cooperative projects with other government 
agencies. Details on many of these programs -can be found in the February 1962 
GOSC and APR reports. The lauditory comments and spirit, of cooperation evidenced- 
by all in attendance at these "conferences were in and o*f themselves a testimony 
of DLIFLC progress across a whole system of training activities. ^ 

* . Chapter 1 . \. . /" 

Page 2 : (1) The DLI provides training in 37 major languages and dialects. 

(2) The^ DLI detepnines xourse^lengths in coordination with the Services 
and user agencies. 

(3) The Commandant maintains administrative control of all Army stud?rfPs 
at the DLI. * , ^ 

Page 3 : The 1980 GAO report on foreign language needs neither evaluated, ' 
analyzed, nor reported on extensive data provided by the Department of 
Defense concerning Defense foreign language requirements and capabilities. 
The Defense information was provided en topo to^th^ interestPd Congressional 
Committees at DoD request and was only used by*GAO to determine aggregate- 
totals. It is completely misleading and false to reference the 1980 report 
as shedding any light on DoD linguistic competence or on the value of 
' training programs . 

Page 4 : We have no record that the 1^79 and 1981 DLI external field evaluations 
were generally acxepted by Service officials. 

Ch^ter 2^ 

In^the area of course development, resident course development has and will 
continue to take precedence over nonresident course development. However, 
nonresident projects wij^ continue to be supported because they are needed 
to refresh, maintain, and improve the hand-won skills acquired in resident 
training courses. In short, they are iieeded to protect our linguistic invest- 
ment. As- of March 1982, 74 percent oT course development resources ar£ allocated 
for resident programs; 26 percent for --nonresident and refresher/maintenance 
programs. . ^ * 

DlIFLC*s priorities for course development are set by DoD user agencies (NSA^ 
and the Services). The current system for establishing project priorities 
through the TDFYP is efficient and effective and satisfies the needs of DoD 
user agencies. The TDFYP was again confirmed 'by all OoD user agerw:ies during 
the 'APR of Feb 82. Other user agencies outside DoD also expressed confidence 
in the current system. Principals clearly confirmed ^that resident programs 
have priority over nonresident prograijs, that signal intelligence requirements 
have priority over other neetfS, and that potential '"tl^eat" languages. have 
prTbr*ity over'"non threat^Manguage. DLIFLC is meeting the needs of DoD user 
agencies in response tg stated and approved priorities and has recently 
initiated a s^ies of production control measures to improve course developm^t ^ 
programs. 
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DLIFLC is currently developing resident basic courses in Russian (will be 
completed in Sep 82, a validation edition is presently being taught in the 
classroom).. Korean. (55% completed). German (10% completed) . Arabic (Modern 
Standard) and three dialects; Egyptian 85% ce-^Pl^i^J)' Syrian 85% completed), 
and Spanish is also under development. Analysis and ^esgn package for all 
Basic Courses presently under development have been co-npleted Analysis and , 
design are also being initiated for new Italian. Greek 

Courses. The above listed programs were approved and funded Tn the TDFYP and 
represent all high density "thrwit" languages. 'In addition. DLIFLC is contin- 
uously updating all Basic Courses through an established course maintenance 
program within the individual departments under the overall supervision of a 
newly developed position of language maintenance coord inato^r., 

In^ame«rt with our course develop<nent efforts DLIFLC filT not only adopted 
an instructional systems development approval, but has also contributed to the 
state of the art in foreign language curriculum development. DLIFLC also 
established, in late 1981. a program management which plans for and obtains 
adequate resources, and monitors and assesses the progress of all development 
projects. The Commandant and Academic Dean are provided a quarterly briefing 
on the status of each workplan. 

While it is true that DLIFLC could have better managed course d/velopment 
resources in the past, performance in this area should be vie)/ed with an 
understanding of the massive effort associated with the development of a 
single new Basic Course. Our new Basic Courses provide in one week the number 
of instructional hours in a quarter length college course. Furthermore, 
development goes far beyond just the preparation of classroom teaching "jaterials 
to tests. Instructor handbooks, home materials, etc. Multiply these major efforts 
times the number of high density "threat" languages and one can better understand 
the nature of manpower expenditures involved in the DLIFLC course development 
program. 

DLIFLC has a long-standing policy of reviewing commercially available text 
materials-^or possible use inits curricula and has used and adapted commercial 
materials for several courses from time to time. The limited scope of these 
materials, copyrig'ht restrictions, frequently exorbitant prices, and uncertain 
availabilities are but a few of the factoes which resulted in a conscious 
decision to "develop" rather than "buy" course materials in the past. Not- 
withstanding. 'these difficulties and uncertainities . DLIFLC still pursues the 
acquisition of appropriate commercial materials where appropriate. It should 
also be noted that CIA. NSA and FSI language schools al?o rely .primarily on 
government-produced materials in their own high density language program basic . . 
courses. 

«• 

Similarly, because non-government agencies have not 'had experience developing . 
courses of the magnitude required for intensive DLIFLC training, we have learned 
to proceed with caution in contracting course development projects. Of course, 
where the required expertise exists, we will continue to maintain the ^ption 
of developing materials under contract as we are now doing with several test 
development projects. 



r 



APPENBIX I • APPENDIX I 



Chajfter 3 



DlilFLC regards professional faculty development as the cornerstone of DLIFLC 
training. We have been doing a lot In this area and will be doing even more. 

Although the GAO report credits only "a few Instructors" as having completed 
faculty training aside from the BITW Phase I, actually 506 faculty members 
received training during the period Jan 80 to 32 Feb 82; training subject matter 
and number of faculty In attendance are set fofrth below: 



Course Description 

Criterion Referenced Instructlon(CRI) 

Criterion Referenced Tests- (CRT) 

CRT for Managers 

CRT for Item Writers ' 

English for Professional Development 

Group Dynamics & Leadership 

Introduction to Linguistics 

Audio-Visual Training 

Counseling 

Intro to Instructional Systems and 
Development and tesson Design 
Val Idatlon Course 

Appl led Linguistics 

Total 



•May 81 



36 
29 
6 
3 

33 
29 
81 
58 
22 
5 



31 



Jun 81 -Feb 82 
9 

12 
0 
0 

, 34 
0 
20 
19 
58 

4 / 
15 



335 



171 



only 77 percent of newly 
ning - Phase' I," nearly 
Basic Instructor Trainin 
resent. The discrepancy 
the fact that only^about 
coyrse of instruction, 
following completion of 
nguistics course is not 



hired instructors as receiving 
95 percent of newly hired instructors 
g Workshop (BITW) Phase I during 
in the two percentages appears 
70 percent of BITW Phase I graduates 
Introduction to Linguistics, 
Phase I training. Internal review 
appropriate for beginner instructor 
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While the GAO reports 
"Basic Instructor Trai 
received the two week 
the period Jan 80 to p 
to. be attributable to 
completed the one week 
presented immediately 
has found that this 11 
training, 

DLIFLC has begun al^ajor revision of the entire DLIFLC faculty professional 
development program, based oa data identified during a Faculty and Staff 
Division Instructional Systems Audit (ISA) and follow-on DLFLC Faculty and 
Staff Division Ta'Sk Force, and the more recent detailed review by the Academic 
Dean. 

Plans have also been made for a detailed review of DLtFLC facility and staff 
courses and curricula by a team consisting of national -known visiting professors 
and selected members of the DLIFLC teaching 'and management staff >*ho are well 
versed in professional faculty development. Extensive review ,-4/i'd revision of 
current faculty professional development course materials w(Tl. t^ke place in 
the summer of. CY 82, ^A thorough review and revision of other. facul ty develop- 
ment materials above and beyond BITW will also be initiated". Efforts in this* 
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regard will be directed toward several major areas: basic teacher training, 
developing English language proficiency of the faculty; BITW reinforcement 
•training ope year following initial hire, supervisor and chairperson managerial 
training; refresher training for high tenure non-supervisory instructors, and 
expansion of audio-visual training in response for increased use of audio-visual 
technology in the classroom. 

While teaching methodology will be part of this total review, it should be 
mentioned that dIiFLC presently has an eclectic approach to language learning 
comparable to that used by CIA, FSI, and NSA. Our failure has obviously been 
in not communicating t^ GAO and some members of our own faculty and staff that 
this eclectic approach is indeed a methodology. One of the most immediate 
difficulties in this -regard- is a problem of nomenclature. It is semantically 
impossible to talk about "the eclectic method." 

Ovlrall management of the Faculty and "Staff Development Division has. been 
piked directly under the Academic Dean's office to insure that full attention 
is given to -this most important aspect of DLIFLC's training mission. 

DLIFLC ha^ revised post-Instructional Systems Audits and Field Evaluation 
report procedures. These changes require that the Directorate of Evaluation 
not only provide formal ISA ajid Field Evaluation Reports, but also a personal 
briefing to the Academic Dean, concerned Directorates, Group Chiefs and -language 
Department Chairpersons. Additionally, the Directorate of Evaluatfon also now 
provides its recommendations to others at the post-ISA/Field Evaluation meetings, 
These recommendations are then discussed at lengthyuntil a consensus is reached 
as to which are practical and feasible. This refi/esents a change to procedures 
in effect during the period of the GAO report. At that time, the Directorate 
of Evaluation did not discuss its recommendation with concerned parties, with 
the result that often they were not well received and in turn, not followed 
through. The revised procedures have improved communications an* led to the 
identiftcatton and implementation of more reasonable recommendations. The 
Academic Dean personally chair§ the post ISA/Field Evaluation meetings and 
personally monitors execution of recommendatiofis made. 

The Directorate of Training, acting under direction of the DLIFLC Commandant, 
has launched a formal Defense Foreign Language Professional Development Program 
as a vehicle designed to improve management of technical language assistants 
in response to DLIFLC and DoD user agency needs^ 

Chapter A . - * • 

DLIFLC has as cohesive a set of standards as any U.S. Government "agency, 
although they could be more specific; as a matter of fact, the Interagency 
Language Roundtable UlR) which includes DLIFLC, NSA, CIA and FBI, is presently 
actively reviewing and will Republish U.S. Government proficiency standards, 
which, when ^Wished, will be the benchmarks for DLIFLC training objectives. 
DLIFLC is playing an active and leading role in these ILR actions, and is 
working in close concert with all ILR participating agencies to improve U.S. 
Government st§nd^rds as a whole. Some indicators of the leadership DLIFLC is 
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providing the language conwunlty In the area of language testing are- ^he 
November 1981 National Language Proficiency Testing Conference, hosted by 
DLIFLC. and our active coordinating role In several 1/iteragency test develop- 
ment projects now being pursued. 

Following publication of the ILR-developed U.S. Government proficiency sjamirrds.^ 
DLIFLC will revise its DLIFLC Memorandum on this subject to incorporate the spirit, 
thrust and intent of these new standards. 

^ - 

DLIFLC's mission is to provide general language training. DLIFLC's mission 
is not to prepare students for a specific mission, but rather, general language 
missions. DLIFLC's role in the foreign language learnirjg process is to prepare 
its students for a wide variety of language-related occupational skills; for 
all of these jobs, language is a necessary, but not sufficient qualification 
for accomplishment Of the task. Language is common to all of these occupational 
skills Even NSA has gone on record stating that DLIFLC's mission is to 
prepare its students for general mission tasks, while NSA and Services are to 
provide specific job-related skills language training at follow-Dn MOS training / 
facilities.' 
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DLr STUD'ENT enrollment h^D INSTRUCTORS 
BY LANGUAGE AS OF SEPTEMBER lU- 1981 

Students 



Language 

• 

Asian/Middle East group: 
Arabic 
Chinese' 
Tagalog o 
Greekv 

Indonesian (including Malay) 

Japanese 

Korean 

Persian 

Thai 

Turkish 

Vietnamese 

Total 

Rpmantic/Germanic 'g^oup: * ^ 
Albanian 
Dutch 
Fttench 
Hungarian 
German 
Italian 
Norwegian 
Portuguese 
Romanian 
Spanish 

Total ^ 



r 



Iris 



tors 



186 




35 


196 




37 


2 


■ 


1 


• 31 




8 


7 - 


• 


3 


10 




2 


361 


» 


62 


' 6 




3 


3 • 


* 


3 


. 36 ■ 




« 11 


49 


• 


8 « 






173 


2 




1 


■ 4 . 
•23 




1 
9 


7 




3 


304 




.55 


3d 






4 




1 


6 . 




4 


7 




2 


182 




28 


577 




112 



Slavic group: 
- Russian 
Bulgarian 
Czech 
Polish 

Serbo-Cr6atian 
Total 



Total 



a/1,213 
5 

134 
70 
8 

.1,430 



2,894 



J. 



a./ 179 
3 

' 20 
15 
- •6*' 

b/508 . 



a/Includes 410* students and >55 faculty members at Lackland. 

b/ln additi.on, there are 389 secretarial and 68 rtianagement 
~ staff. , • ^ ' ' 
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. . PREVIOUS STUDl^SyOF OPERAMONS 

— 

» AND .TRAINING AT DLI ^ 

General Accounting Office, Ueed To Improve Language Training 
Programs and Assignments for U.S. Government Personnel 
Overseas, B-176049, January 22, 1973 

General Accounting Office, Improvement fJeteded in Language Train- 
ing and Assignments for U.S. Personnel Overseas, ID-76-19, 
June 16, 1976 ^ - ^ . 

General Accounting Office, Need to Improve Foreign Language 
Training Programs and Assignments for Department Defense 
Personnel, ID-76--73', November 24, 1976 

General Accounting Office, More Competence in Foreign Languages 
Needed by Federal Personnel Working^Overseas , ID--8Q-031, 
April 15, 1980 

Army Training and. Doctrine Command, Defense Language Institute 
Assessment, August 12, 1975 

Defense Language Institute, Accreditation Self-Study Reports 
June 1978 . . 4 

United States Army Intelligence School, Faijrt Devens - Defense 
Language' Institute, J'oreign Language Center, Joint Field 
Evaluation, Far East and Germany, 1979 

Defense Language Insti*tute, Foreign Language Center, Field Ev^i- 
uatibn of DLIFLC Graduates, European Command, April 24, 
1981' * 1^' 

President's Commission on Foreign Language and International 
Studies, Strength Through Wisdom: A Critique of U.S, 
Capability, Noventber 1979 ' ^ :^ ' 

Army Training and Doctrine Command Inspector General, RepcSrt 
of Findings, March 2, 1979 ^ • ; 

Department of the Army, T^ie Army Linguist Problem, Aprife 24, 19^ 

Defense Language Institute, Special Program Review, Summary 
Report, December 1980 

Defense Language Institute, , Annual Prog tarn Review, Summary 
Report, February 1981 

Defens* Language Institute, Foreign Language Center, General 
Officer Steering Committee, Report, January 30, 1981 

N 
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Army Training and Doctrine Command r Defense Language Institute, 
Foreign Language Center, Management Study, July 1981 

Department of the Army, MILPERCEN. Linguist Survey, 1977 

Department of the Army, Army Linguists Personnel Study, 1976 

Department of the Air Force, Study^of Voice Processing Linguists, 
1979 ' ' 



33 



ERIC 



13 



APPENDIX IV 



APPENDIX IV, 



LMirwfe Skill Level beMriptkme 

is ■ 



tM^iw rtT AM* U M tot iiJ 
pto «rMtiMt <i»dii«af boc^ 



Abto to MtWy rmte trtv«l 
4|irirm»to. Cm uk va^^mmm 



to A/fear wlihiii tto 



No pTMtteal wriUaf praOdMcy. 




IM far iJowW •fH^ 
or pvifkrvr i pMMit m^tt- 
\m1 toiHuMti to «ipr«M ab7- 
thtef bat Mt itow wto ry 
«mn in pr<#«dBUM Md 
.frmMU^ in fr«f«Mt. bat eu b* 
Mdmtood fay • tttlv* ■Pw faT 
M to Ml^ wiU forMfm at- 
tmptiaff to ipMJt Ut/lMT IM- 
(Wf*-> «^ tofka vbieh m 

*rM«l''to MhrMwt. _ . 
«t thte Itirri ibMlChrabl* to «r4ar 
• ilapto mL wk far Mur or 
^ IWgtac. uk ud givt iteplt dkM- 
ud ton 



priatad miM W » alybaWtk 
•7«t«i ABd bigb» en q » a «c y tla- 
MU af • qrOftbtfy tr • ebMartar 
•ritaB. 



•^fliiaat aastial oTUm writ- 
iBf kyMi to aaat ttaltad prM- 
tkal oaadt. Cm pr»d«ea aU ay*- 
bote ta M alpktbatk or ijMt 
writiH Cm wrtto 

bora ud dstaa. Ua ovi naiM Md 
MtioMUty. >ddr n i H . ate Otbar- 
wlaa. abfllty to wrtto ia Umitod to 
rimida UaU of eouaa ttosu or « 
{o« abort aostaoeoa. Spa&iM 
baarratie. 



AUa to Botfalt ro«ilM aadal do- 



^tmtt. Cm kaadl* wlth^eaaB- 
daM*. b«t oat wfU facility. MSt 
■mW ittaatioaa. taditdiH 

Sm Md canal ^oy a n i ti ooa 
cvraait ovaota, oa «*fl ao 
baaOy.aad aatobtecnpUaal 



fraaa^faio 

wttbU a fa- 
oxtoaliira oaa 
got tba raOoral 
haaiiiai lottara. 
l^Unatiaoai oovt itom. or artl 
efaa faloebateal Salda wItUa bfa/ 




bolp 

to Mdtof My aaa pHrotfa M «t 
lot tW flat ¥ 



Cm Mt rooilM aocfal, eoiro- 
mna d a i r i aad naot JMtod. pre- 
fa^aiooai oooda. la frmW** witb 
tba mrr^**^ of tb* wrftiof aya- 
imm. axeoft fa cbaraetor aratofM 
wb«r« aMMty la tteHod to « aaaO 
flCocfc of Ufb^a^oanq »toaM. 
liabaa froqooot arren fa apoOfaf- 
•Cyfa. aad wHtfaf eoo^tioM. 
AMo to writo ttmnf^ aot« Md 4 
draft roattea aoefal Md Ifaittod o«- 
flea M^Mcaa. Matartol aoraMDy 
rosakaa adWaf by a aaro 



aal a^jaeto tor^a wbfab ro- 



ataiplyiaaai 
ocaMrt, tMfb «<Ua 9lidu faalty. 
fa fatoOgMr. (€U aoaoBy baadfa 
aHnaHryaaaotmtlAaqartoae- 
onuly bat dool aoi ba«« tbar 
eoatrol of tba 



, ' 0 
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Unguff* SU« Leve* DetcripHoM-Contiotted 



Ate to MiwvUM U« 



MNlj hM to Mk f» pirafkniteff 



AU« to ifMk tl» taHWf* wAh 

MffiiM Knetinl tcevMy 

i. aoit famal m4 tekraW 

c«M ptfMv iBtor«U ftiii an- 
to 

yjto B Bf ^ < tor • KWMl rrt« 
^ 4pMck; Mbirivr to brMd 

■tr«l •! 1 
mr toUrltn wHk 



Mk>«to; cs^ Mitntud wttM 
«iMl7 a torn lUiiM 



Ay* t^ntfinUW iB torvt ud 
itjiM •! wtmn^ p«tto«^ to pfo- 
fiTr" — • Abto U • 

fiMi MIy aB i^Mck to all Md- 

tH dtotoeto w M7 Mt^JM^ 
vttt to praiMteMl MMto «ttUa 
U« nag*, W Us>te MywtoM*, 



«d CM»i<natto«r 

toVMCMU Md t 

pliM* cdto; nd afl ktodt of toch- 



■kil 
Abtoto 
•iMckto 



CaapftfciMi'fli •quinkst to tkat 
of the vdwatod Miivt i9««k*r 
AW» to udOTiUad Ui^ afi ianu 
MMl ktyloa ol tpMch fUUsfiU* to 
the edueatod saUv* ap«ak«r. m 
dttdiaf « apAbM- of rvfMui and 
tUit«r»to dtolMcU. lufiitr eoito 
spooeh. a»d cDiiv«Mftt»ont 
ud diMo^ distortod amrkcd 
iBtorforeaco tro« oibor bom* 



AM* to road iUiidard nowtpaper 
idftaaad to tho foaoral 
rowttoa eorwipoadeaot. 
rcperu aad toekakal natonal to 
Ui/terapactolltold Cuigraaptho 
Mtwtiih al arUdoa o< tka abov* 
iTpoa wiUwOt aiiof a dicUoMnr. 
(or aeeirato wdaraUadiaf . 9ad< 
oratoly fr^niMrt •** of a dicttoa 
„j It ro^ttifW. Haa ^eaaiool dif- 
Set^y witk waai^ny ««»ptox 



AM* to l*a to»fiiaffa BoaBti^ 
and acCTrataty oa aB to««to aar 
mDj porltooat to prvMml 
Mad» Caa Mdarauad a»d partk- 
ii aay eap^anaUoa wit«a 
iha raafa W Ua axpartoaca wttk a 
Ufk dagm W fiaaKT aad prad- 
liea of Tacabatorr: «aa>d raraljr 
ba tokaa far a aa^jra ^aakar. bat 
<aaraapaadapprep ri a t a iy wato 
MiaaOBar il t a at toa a: arron.of 
ptaaawtottoa aWifraaia^ qaito 
nrr cm baadU tofarsat tator- 
prattof &«■ a^ toto tba toa 



Caa draft offieial «^ ■ 
ud raports la a apodal HaU. Cea:^ 
trd ol rtmctBTt. apaUtog. aad 
voeabvterr la ado^aaU to eaavay 
Ui/bar ^aaaaga aeeuratoly. bt< 
■lyte ougr ba qiuto laraiga. Afl lor- 
■al wriUaf aaada to ba adMad by 
aaadaeaUdaatfra. 



Abia to road aB aiy to* and foma of 
tb* toagnaga pertiaaat to praf*a- 
aioaal a**^. WKb aecasmul oa* 
of a AeUoaary eaa road madarato- 
ly difflcolt praaa raadily to aay 
ana dimud to ibo seaaral raad- 
or, aad aB autonal to bttf/b«> 
dal Oald. lododtaf *0total aad p»f- 



tprmrkarti caa raad r aa aoaahiy 
logiWa baadwrWag whhoal dim- 
ealty. 



Can draft aD toraia af proa* parti- 
!*•< to pfa«aa*ioaal a**di. CaaU^ 
of atractor*. voeabatory. aad apaB- 
li( ta broad aad i^aetor. aaaaraf 
•tyto to aaariy aatlva. brara ara 
rara ^ da BOt latorfaro wWraa- 
darauadb^. NoirortWtoaa, dralto 
of ofBdal earroapeadoa«a aad dBC> 
Moat* aaad to ba aditod by aa 
idatatad lattv ' §^ 



a^oaJdag proQbaaey aqvhratoat to 
tbai of aa adaeatod aativo ipaak- 
«r. Haa ea«ptoto flaaacy to tb* 
laaftttg* aacb tbat toa apncb oa 
aU tovoto ia faUy aeeirptod by 
odaeatod aativo ipoakar* u all of 
t(a faatwaa. tadodtof broadtb^ 
vocaboUry aad idtoai« eoUoqaul- 
uou. tad partiaaat catioral raf 



JUadtof proAdaaey oqvhratoat to 
U«t a( aa adaeatod aatlr* Caa 
raad oxtraoMty dUfieaK aad a^ 
itra«t praaa. a* waB aa UgUy aal- 
loqi^ wrHtop aad tba daaak 
fatorary torw of ibo Uagaag*^ 
WHb raryiai dagraoa of diffieatty 
caa raad aO kiad* of baadwrittoa 



Haa writiag proOdaaey aqaal to 
tbat af^aa adaeatad aathra. Caa, 
draft aad adit beib laraui aad to- 
formal aarragpiada aw . offidal 
roporta aad doeaaaata. aad pro- 
faadoail artktoa wttbaat oaa- 
oativa artari of ttraetara. *p*U- 
tng. atyto« or vocaboUry. Acbtovo- 
aaat «f tUa raUag aoraally ra- 
q^rai botb Mw a d ary aad bigb 
odacattoa m tofOtaUoaa wbar* tba 
Uagaaga to tba ^riaary aaa aaad 
(ar toalrarttoa. « 
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Language Proflciency Code Key 



\ 



C-1 VMitsUry Prof: 
UiidmUBdt Mfl tfrnptt 
qotflUM tad fUttmtBt* 
M fftinOUr topics whts 
tpok» to vtry slowly and 
tfisUoct^.Tbtseoftsn 
iMvt to bt roststed ia 
aiff8r«l^)«f<^b«foroho 
oadorstuds. 

C-2 Lfanltsd WorUof 
Prof: 

Uadsrstaods moot coovcr- 
.stUoo whof spokoB dis* 
tiactlysodstislowor 
tksa oomsl rtU. PoioU 
havt to bo rostotcd oeco- 

flOMflf. 



C*3 Minimum Teth Pr^: 
UodonUods genersl coa- 
vorsstkm or discossioo 
witUa'ldsspodilliold. 
wkm tim rsU s psodi is 
MOT oornML 



C-4 rrilTodiProf: 
UadorsUods say cooTor- 
sotioo witlda tile roan of 
bis oxporioaeo wboa of 
■oroisl coovorsstioBsl 



C-ft NotivoorBiUafoal 
ftofld oocy: 

C om p f o boa slo B proCdoa- 
C7 o^iMoai to tbot of SB 
odotttod BsUvo spoakor. 



8-0-NePrscUcil 

m> I flgliHiw 
rrODaODC/ 

8-1 ElomsBtWT^i^: 
Asks SBd snswors quoi- 

tioBs oa dsUr persoBsI 
Botds. withio s limHod 
vocsbulsry sa4 with fro- 
quoBt orrors ia pronuods- 
UoB sad frsmrosr. 



8-2 Umit^Workiaf 
Prof: 

CoBvtrset iatolHffblr bat 
wttboot tborouf b eootrol 
of proouacUtioo sad 
frammsr within host 
soeisl situstioos. sbout . 
eurrcat tvents. his work. 
Ismtiy. sutobiofTSpbicsl 
{nlormstioo and noa- 
tochnics) subjects. 

S-3 Minimum Tech Prof: 
Partieipsies effectively in 
«n pneral conversstion. 
discitssos psrticuUr ia- 
' icrcsU.sndbisspocial 
IMd^wtthotAfflskiBf 
«fTors tbst obscure mesn- 
i>C. 

S-4 FdlTocbProf: 
Spesks tko Ua(usffoih»- ' 
oatly sad sccurstely oo sll 
Wrels portlaont to mili- 
tary service needs with- 
•m orrors of proooada- 
floo or frsmmar thai 
ifttorforo with ease of 
MdorstaadiBf. 

8-5 Native or BOiafoal 



R^-No Practical 
ProfleloBcr 

|t-l OomoataryProf; 
Roads aa^ uador^aads 
tlemealsry lesson msto- 

risi snd common public 
signs. 



R-2 Limited Working 
Prof: 

Resds snd understands ia; 
tormediste lesson rasto- 

rial or sim^e colloquial 
texU. ^ 



Spesks witk s profieieacy 
oqiavaleat to tbst of sa 
odacaiod boUvo i^kcr. 



R*^ Minimura Tech Prof: 
Resds snd understands 
material on military and 
iateraa t ional subjects 
wHhia his field. Resds sad 
undorstsBds techmeal 
text mst^al at junior 
kigb scboel lovol: 

R-4 FttU Tech Prof: 
Reads high school level 
prose and material ia his 
special Tield sod military 
documents snd corre- 
spoadoaeo. 



R S NMive or BtUagttsl 
Profidoncyt 

Reads at a level of ^rofi- 
cieocy equivalent to that 
of aa educated aattvo. 



W-O-No Practical 
Profidoacy 

W-1 Semoatsry Prof : 
Wrftos simple sUtemoats 
sad questions using s very 

limited vocsbulsry with 
frequent errors^ spelling 
sad structure that fre- 
quently obscure meaning. 



W-2 Limited WorVing 
Prof: 

Writes sentences on famil- 
iar topics using s technical 
vocabulary and basic 
structure psttern. Errors 
in spelling snd structure 
occssionslly obscure 
Kosniag. 



W -3 Minimum Tech Prof: 
Writes psrsgrsphs on 
famOiar topics using oon- 
technical vocabulary and 
basic structursl patterns. 
Errors seldom obsctro 
mesatng. 



W^ Pun Tech Prof: 
Writes prose with sufC- 
cieat structural acAracy 
aad vocabulary to ssttsfy 
pertinent service re- 
qoirottoata* 



W-5 Nstive or BillngusJ 
ProficioBcy: 

Writes with s Proficiency 
eqoivslent to thst of so 
educated asUvo spesker. 
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